import SetupEnv from './common/setup-env.mdx';

# Bridge mode by Chrome extension

import { PackageManagerTabs } from '@theme';

The bridge mode in the Midscene Chrome extension is a tool that allows you to use local scripts to control the desktop version of Chrome. Your scripts can connect to either a new tab or the currently active tab.

Using the desktop version of Chrome allows you to reuse all cookies, plugins, page status, and everything else you want. You can work with automation scripts to complete your tasks. This mode is commonly referred to as 'man-in-the-loop' in the context of automation.

![bridge mode](/midscene-bridge-mode.jpg)

:::info Demo Project
check the demo project of bridge mode: [https://github.com/web-infra-dev/midscene-example/blob/main/bridge-mode-demo](https://github.com/web-infra-dev/midscene-example/blob/main/bridge-mode-demo)
:::

<SetupEnv />

> In bridge mode, the AI models configs should be set in the Node.js side instead of the browser side.

## Step 1. Install Midscene extension from Chrome web store

Install [Midscene extension from Chrome web store](https://chromewebstore.google.com/detail/midscene/gbldofcpkknbggpkmbdaefngejllnief)

## Step 2. Install dependencies

<PackageManagerTabs command="install @midscene/web tsx --save-dev" />

## Step 3. Write scripts

Write and save the following code as `./demo-new-tab.ts`.

```typescript
import { AgentOverChromeBridge } from "@midscene/web/bridge-mode";

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
Promise.resolve(
  (async () => {
    const agent = new AgentOverChromeBridge();

    // This will connect to a new tab on your desktop Chrome
    // remember to start your chrome extension, click 'allow connection' button. Otherwise you will get an timeout error
    await agent.connectNewTabWithUrl("https://www.bing.com");

    // these are the same as normal Midscene agent
    await agent.ai('type "AI 101" and hit Enter');
    await sleep(3000);

    await agent.aiAssert("there are some search results");
    await agent.destroy();
  })()
);
```

## Step 4. Start the Chrome extension

Start the chrome extension and switch to 'Bridge Mode' tab. Click "Allow connection".

<p align="center">
  <img src="/bridge_in_extension.jpg" alt="bridge in extension" width="400"/>
</p>

## Step 5. Run the script

Run your scripts

```bash
tsx demo-new-tab.ts
```

After executing the script, you should see the status of the Chrome extension switched to 'connected', and a new tab has been opened. Now this tab is controlled by your scripts.

:::tip
⁠Whether the scripts are run before or after clicking 'Allow connection' in the browser is not significant.
:::

## Constructor

```typescript
import { AgentOverChromeBridge } from "@midscene/web/bridge-mode";

const agent = new AgentOverChromeBridge();
```

Except [the normal parameters in the agent constructor](./api.mdx), `AgentOverChromeBridge` accepts one more parameter:

* `closeNewTabsAfterDisconnect?: boolean`: If true, the newly created tab will be closed when the bridge is destroyed. Default is false.

## Methods

Except [the normal agent interface](./api.mdx), `AgentOverChromeBridge` provides some other interfaces to control the desktop Chrome.

:::info
You should always call `connectCurrentTab` or `connectNewTabWithUrl` before doing further actions.

Each of the agent instance can only connect to one tab instance, and it cannot be reconnected after destroy.
:::
### `connectCurrentTab()`

Connect to the currently active tab.

* Type

```typescript
function connectCurrentTab(options?: { 
  forceSameTabNavigation?: boolean 
}): Promise<void>;
```

* Parameters:
  * `options?: object` - Optional configuration object
    * `forceSameTabNavigation?: boolean` - If true (default), restricts pages from opening new tabs, forcing new pages to open in the current tab to prevent AI operation failures due to manual tab switching. This configuration usually doesn't need to be changed

* Returns:
  * Returns a Promise that resolves to void when connected successfully; throws an error if connection fails

* Example:

```typescript
try {
  await agent.connectCurrentTab();
  console.log('Successfully connected to current tab');
} catch (err) {
  console.error('Connection failed:', err.message);
}
```

### `connectNewTabWithUrl()`

Create a new tab and connect to it immediately.

* Type

```typescript
function connectNewTabWithUrl(
  url: string,
  options?: { 
    forceSameTabNavigation?: boolean 
  }
): Promise<void>;
```

* Parameters:
  * `url: string` - URL to open in the new tab
  * `options?: object` - Optional configuration object (same parameters as connectCurrentTab)

* Returns:
  * Returns a Promise that resolves to void when connected successfully; throws an error if connection fails

* Example:

```typescript
// Open Bing and wait for connection
await agent.connectNewTabWithUrl(
  "https://www.bing.com",
  { forceSameTabNavigation: false }
);
```

### `destroy()`

Destroy the connection and release resources.

* Type

```typescript
function destroy(closeNewTabsAfterDisconnect?: boolean): Promise<void>;
```

* Parameters:
  * `closeNewTabsAfterDisconnect?: boolean` - If true, the newly created tab will be closed when the bridge is destroyed. Default is false. The will override the `closeNewTabsAfterDisconnect` parameter in the constructor.

* Returns:
  * Returns a Promise that resolves to void when destruction completes

* Example:

```typescript
// Destroy connection after completing operations
await agent.ai('Perform final operation...');
await agent.destroy();
```

## Use bridge mode in YAML script

[Yaml scripts](./automate-with-scripts-in-yaml) is a way for developers to write automation scripts in yaml format, which is easy to read and write comparing to javascript.

To use bridge mode in yaml script, set the `bridgeMode` property in the `target` section. If you want to use the current tab, set it to `currentTab`, otherwise set it to `newTabWithUrl`.

Set `closeNewTabsAfterDisconnect` to true if you want to close the newly created tabs when the bridge is destroyed. This is optional and the default value is false.

For example, the following script will open a new tab by Chrome extension bridge: 

```diff
target:
  url: https://www.bing.com
+ bridgeMode: newTabWithUrl
+ closeNewTabsAfterDisconnect: true
tasks:
```

Run the script:

```bash
midscene ./bing.yaml
```

Remember to start the chrome extension and click 'Allow connection' button after the script is running.

### Unsupported options

In bridge mode, these options will be ignored (they will follow your desktop browser's settings):
- userAgent
- viewportWidth
- viewportHeight
- viewportScale
- waitForNetworkIdle
- cookie

## FAQ 

* Where should I config the `OPENAI_API_KEY`, in the browser or in the terminal?

When using bridge mode, you should config the `OPENAI_API_KEY` in the terminal.





